Prepare for using SQLite on AWS (backup on S3) #81

nolan1999 · 2026-02-04T15:42:56Z

Support backup to S3 of local SQLite DB. This allows restarting AppRunner without losing all the data if using SQLite as a database. This will make the stack cheaper, since no RDS is required anymore.

Copilot

Pull request overview

This pull request adds SQLite backup functionality to S3, enabling the use of SQLite as a database for AWS AppRunner deployments without data loss on restarts. The changes eliminate the need for RDS, making the infrastructure more cost-effective.

Changes:

Implemented automated SQLite database backup to S3 with hourly backups and backup on shutdown
Migrated from fastapi_utils.tasks.repeat_every to FastAPI's built-in lifespan management for background tasks
Refactored authentication classes into a separate module for better code organization

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
workerfacing_api/settings.py	Added cron interval configurations for timeout checks and backups
workerfacing_api/main.py	Replaced repeat_every decorator with lifespan context manager and added backup cron job
workerfacing_api/dependencies.py	Extracted auth classes, added S3 client setup, and implemented conditional queue instantiation (SQLite vs RDS)
workerfacing_api/core/queue.py	Added SQLiteRDSJobQueue class with backup/restore functionality, refactored RDSJobQueue with lazy engine initialization
workerfacing_api/core/auth.py	New file containing extracted authentication classes (APIKeyDependency, GroupClaims, WorkerGroupCognitoCurrentUser)
tests/unit/core/test_queue.py	Updated test to work with simplified create() signature
tests/integration/test_main.py	Added comprehensive tests for timeout handling and backup/restore functionality
tests/integration/endpoints/test_jobs_post.py	Updated fixture scope from function to default
tests/integration/endpoints/test_jobs.py	Refactored tests to use isinstance checks instead of env string comparisons
tests/integration/endpoints/test_files.py	Refactored tests to use isinstance checks and updated fixture scopes
tests/integration/endpoints/conftest.py	Added client fixture with proper lifespan context manager
tests/integration/conftest.py	Restructured test fixtures to support parametrized filesystem and queue combinations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

workerfacing_api/main.py

workerfacing_api/core/queue.py

tests/integration/test_main.py

workerfacing_api/core/queue.py

tests/integration/test_main.py

github-actions · 2026-02-04T21:04:44Z

Coverage Report

File	Stmts	Miss	Cover	Missing
workerfacing_api
__init__.py	0	0	100%
dependencies.py	33	13	61%	13–19, 32, 36, 60–71
exceptions.py	4	0	100%
main.py	52	4	92%	26–27, 40, 83
settings.py	30	5	83%	8–11, 35
tags.py	1	0	100%
workerfacing_api/core
__init__.py	0	0	100%
auth.py	21	5	76%	16, 28–33
filesystem.py	73	5	93%	17, 23, 27, 33, 108
queue.py	345	31	91%	63, 68, 73, 87, 92, 157, 230–231, 256–257, 263, 280–281, 341–345, 354–355, 385, 445–447, 466, 544–546, 555–557, 583, 585, 633
workerfacing_api/crud
__init__.py	0	0	100%
job_tracking.py	11	1	91%	26
workerfacing_api/endpoints
__init__.py	0	0	100%
access.py	9	1	89%	24
files.py	23	1	96%	24
jobs.py	67	12	82%	73–74, 95–96, 149–150, 154–155, 175–176, 180–181
jobs_post.py	9	0	100%
workerfacing_api/schemas
__init__.py	0	0	100%
files.py	6	0	100%
queue_jobs.py	52	0	100%
rds_models.py	32	0	100%
TOTAL	768	78	90%

Tests	Skipped	Failures	Errors	Time
167	26 💤	0 ❌	0 🔥	16m 32s ⏱️

Copilot · 2026-02-05T17:15:34Z

@nolan1999 I've opened a new pull request, #82, to work on those changes. Once the pull request is ready, I'll request review from you.

nolan1999 · 2026-02-05T17:31:25Z

@copilot i get an error in the following test:

pytest -m 'aws or not(aws)' tests/integration/endpoints/test_jobs.py::TestJobs::test_job_files_post[local-queue-aws-fs] -vv
================================================= test session starts ==================================================
platform linux -- Python 3.11.10, pytest-8.3.4, pluggy-1.5.0 -- /home/arthur/pCloudOffline/DECODE_Cloud/DECODE_Cloud_WorkerAPI/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/arthur/pCloudOffline/DECODE_Cloud/DECODE_Cloud_WorkerAPI
configfile: pyproject.toml
plugins: cov-5.0.0, anyio-4.7.0
collected 1 item                                                                                                       

tests/integration/endpoints/test_jobs.py::TestJobs::test_job_files_post[local-queue-aws-fs] FAILED               [100%]

======================================================= FAILURES =======================================================
___________________________________ TestJobs.test_job_files_post[local-queue-aws-fs] ___________________________________

self = <test_jobs.TestJobs object at 0x70c7568ea390>
queue = <workerfacing_api.core.queue.SQLiteRDSJobQueue object at 0x70c756cdaa10>
base_filesystem = <workerfacing_api.core.filesystem.S3Filesystem object at 0x70c756cbbe50>
base_job = SubmittedJob(job=JobSpecs(app=AppSpecs(cmd=['cmd'], env={'env': 'var'}), handler=HandlerSpecs(image_url='u', image_nam...0205172824/test_user/test_log/1', artifact='s3://decode-cloud-worker-api-tests--20260205172824/test_user/test_arti/1'))
test_username = 'test_user', client = <starlette.testclient.TestClient object at 0x70c755ff5550>

    def test_job_files_post(
        self,
        queue: RDSJobQueue,
        base_filesystem: FileSystem,
        base_job: SubmittedJob,
        test_username: str,
        client: TestClient,
    ) -> None:
        queue.enqueue(base_job)
        res = client.get(self.endpoint, params={"memory": 1})
        res = client.post(
            f"{self.endpoint}/1/files/url",
            params={"type": "output", "base_path": "test"},
        )
        assert res.status_code == 201
        if isinstance(queue, SQLiteRDSJobQueue):
            req_base = client
        else:
            req_base = requests  # type: ignore
        res = req_base.request(
            **res.json(),
            files={
                "file": (
                    "file.txt",
                    BytesIO(bytes("content", "utf-8")),
                    "text/plain",
                ),
            },
        )
>       res.raise_for_status()

tests/integration/endpoints/test_jobs.py:255: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404 Not Found]>

    def raise_for_status(self) -> Response:
        """
        Raise the `HTTPStatusError` if one occurred.
        """
        request = self._request
        if request is None:
            raise RuntimeError(
                "Cannot call `raise_for_status` as the request "
                "instance has not been set on this response."
            )
    
        if self.is_success:
            return self
    
        if self.has_redirect_location:
            message = (
                "{error_type} '{0.status_code} {0.reason_phrase}' for url '{0.url}'\n"
                "Redirect location: '{0.headers[location]}'\n"
                "For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/{0.status_code}"
            )
        else:
            message = (
                "{error_type} '{0.status_code} {0.reason_phrase}' for url '{0.url}'\n"
                "For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/{0.status_code}"
            )
    
        status_class = self.status_code // 100
        error_types = {
            1: "Informational response",
            3: "Redirect response",
            4: "Client error",
            5: "Server error",
        }
        error_type = error_types.get(status_class, "Invalid status code")
        message = message.format(self, error_type=error_type)
>       raise HTTPStatusError(message, request=request, response=self)
E       httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://s3.eu-central-1.amazonaws.com/decode-cloud-worker-api-tests--20260205172824'
E       For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

.venv/lib/python3.11/site-packages/httpx/_models.py:763: HTTPStatusError
----------------------------------------------- Captured stdout teardown -----------------------------------------------
Created final backup on shutdown.
=================================================== warnings summary ===================================================
tests/conftest.py:218
  /home/arthur/pCloudOffline/DECODE_Cloud/DECODE_Cloud_WorkerAPI/tests/conftest.py:218: PytestRemovedIn9Warning: Marks applied to fixtures have no effect
  See docs: https://docs.pytest.org/en/stable/deprecations.html#applying-a-mark-to-a-fixture-function
    @pytest.mark.aws

.venv/lib/python3.11/site-packages/pydantic/_migration.py:283
  /home/arthur/pCloudOffline/DECODE_Cloud/DECODE_Cloud_WorkerAPI/.venv/lib/python3.11/site-packages/pydantic/_migration.py:283: UserWarning: `pydantic.error_wrappers:ValidationError` has been moved to `pydantic:ValidationError`.
    warnings.warn(f'`{import_path}` has been moved to `{new_location}`.')

.venv/lib/python3.11/site-packages/pydantic/_internal/_config.py:295
  /home/arthur/pCloudOffline/DECODE_Cloud/DECODE_Cloud_WorkerAPI/.venv/lib/python3.11/site-packages/pydantic/_internal/_config.py:295: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.10/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================== short test summary info ================================================
FAILED tests/integration/endpoints/test_jobs.py::TestJobs::test_job_files_post[local-queue-aws-fs] - httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://s3.eu-central-1.amazonaws.com/decode-cloud-worker-api-tests--20260205172824'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
============================================ 1 failed, 3 warnings in 4.08s =============================================

The pre-signed URLs for posting work fine on the main branch, so something is wrong in this branch. Note how in the code, some specific stuff is done in the boto3 S3 clients because i had already observed similar problems. Please open a PR to fix this error.

Copilot · 2026-02-05T17:31:34Z

@nolan1999 I've opened a new pull request, #83, to work on those changes. Once the pull request is ready, I'll request review from you.

nolan1999 added 4 commits January 30, 2026 21:55

replace on_event (deprecated)

4a3384e

Cron job to backup sqlite db to S3

8a1544d

Simplify dependencies.py

7cbfdf1

Correct cron + add tests

bd99ff3

nolan1999 requested a review from Copilot February 4, 2026 15:42

nolan1999 self-assigned this Feb 4, 2026

nolan1999 added the enhancement New feature or request label Feb 4, 2026

Copilot started reviewing on behalf of nolan1999 February 4, 2026 15:43 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

temp

e064707

nolan1999 added 4 commits February 5, 2026 00:18

No session-scoped monkeypatch

02144f1

temp

c4bbbd5

task group + skip some tests

ce15b4c

correct (no TaskGroup)

328b26b

Copilot AI mentioned this pull request Feb 5, 2026

Use Python sqlite3.backup() API instead of CLI subprocess #82

Merged

Copilot AI and others added 2 commits February 5, 2026 18:23

Use Python sqlite3.backup() API instead of CLI subprocess (#82)

2ad4d0a

reformat

3c92c54

Copilot AI mentioned this pull request Feb 5, 2026

[WIP] Add SQLite backup functionality to S3 for AppRunner #83

Closed

5 tasks

nolan1999 added 8 commits February 5, 2026 18:55

fix tests, hopefully

351c2cd

print -> logging

2692834

fault-tolerant cron jobs

fd2b701

working rds tests?

bb25cda

temp log failure

f3dba65

verbose test output

6e77253

deprecate SQS tests

d14a112

better debugging

5a9349d

nolan1999 added 2 commits February 6, 2026 12:00

now real fix

fad3960

Bigger time intervals to avoid flaky tests

201696d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prepare for using SQLite on AWS (backup on S3) #81

Prepare for using SQLite on AWS (backup on S3) #81

Uh oh!

nolan1999 commented Feb 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2026 •

edited

Loading

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

nolan1999 commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Prepare for using SQLite on AWS (backup on S3) #81

Are you sure you want to change the base?

Prepare for using SQLite on AWS (backup on S3) #81

Uh oh!

Conversation

nolan1999 commented Feb 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

nolan1999 commented Feb 5, 2026

Uh oh!

Copilot AI commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Feb 4, 2026 •

edited

Loading